The apprentice framework: planning and assessing creativity

In this paper we introduce and discuss the apprentice framework, which we speculate can be used to plan and evaluate computational creativity projects. The framework defines a sequence of phases a system must follow in order to reach a level of creativity acceptable to a set of human judges. It also establishes four aspects of a creative piece susceptible of creative work. We mention some examples from different artistic disciplines. Our work focuses on establishing an environment as well as a team of people and machines to foster, study and monitor the emergence of creativity. On Human and Machine Creativity Assessing creativity in machines has become a prime issue in computational creativity after many systems have been built that exhibit a behavior that can intuitively be considered creative. The domains of such systems are so varied, and the versions of each one so many, that comparing them to one another or with different versions of themselves has become a hard task. Every system that claims to be creative must have criteria associated as to what kind of creativity it aims to achieve. After all, in different domains, different notions of creativity may be established. Even so, recently, several frameworks and models of creativity have been advanced to try to capture a generic or general notion of creativity for computer systems (cite: Ritchie 2007, Wiggins 2006, Jordanous 2012, Maher et al. 2013, Colton et al. 2011). They all propose a practical method to unify criteria across the community so that creativity can be measured in systems from different disciplines of application. Computer programs, so far, are designed to produce valuable things for humans. Therefore their creativity is always assessed against human values or needs. This is an unfair situation since it is very hard to program computers to produce valuable objects for humans when these are not well defined and only they can say whether they are valuable or not. If computers were subject to a survival economy like living things in the planet, as Stuart Geiger suggests (2012), then it would be easier to establish what is valuable for them and hence a process parallel to human creativity could be defined to assess creativity by computers. But, for the time being, computers are still doomed to serve our purposes and their creativity will be assessed by human standards. Creative computer systems are still considered in a separate realm to human creativity in practice. They are often assessed against toy scenarios or their products considered as computational creativity (as opposed to simply creative) to avoid measuring them against human products or creativity. This leaves in the observer the decision of whether the system’s behavior is creative in general terms or could be generalized to reach a state where it could be considered so. In creativity we still don't assign the same expectations to machines and humans. Yet the very concept is defined with respect to the latter. So, either human or machine, creativity is not the same, in the sense that they fulfill the same expectations, or they should be assessed by the same standards. But as computers get more involved in creative processes, it is possible to view them as participants and describe what they do as playing a role in a team (Jones et al. 2012). It is possible to interpret the process leading to a creation as collaboration between humans and computers and assign roles to all of them according to their activities. Our view of creativity evaluation is that, although there can be many axes along which it can be measured that seem to be common to several disciplines, ultimately, actual criteria seem to be elusive, arbitrary, subjective and ever changing. These characteristics of creativity don't seem to be problematic to society and most people accept them. It is when we require precision to measure the performance of computer systems that vagueness is problematic. The only way we have to tell whether a computational system is creative, is by inserting it into a human environment and ask humans to assess whether the outcome of the process is creative in the general sense of the term. Thus a concert composed by a computer, for example, will have to be listened by the same group of human experts who would decide whether a composition made by a human is creative, in the general sense applied in music. In this text, we describe a framework we call the apprentice framework to plan and evaluate CC projects. It derives from our multidisciplinary experience in an ongoing project called e-Motion (Negrete, S. & Morales, N. 2013), aimed at building a creative system to produce animatics. These animated shorts, precursors to a final animation, are an essential element of the overall creative process. In the project we examine the relationship between a computing system and the human counterparts that collaborate with it within a successful, creative team. Where is Creativity? Creative products are the result of creative processes. These can take infinite forms but we identify four aspects of creations (creative pieces). Aspects are properties of creations that may be the result of creative work. They are identifiable as the results of separate mental processes that may have occurred at separate times, and may have even been performed by different people: Structure is the basic architecture of a piece; it is what allows spectators to make out different parts of it, to analyze it to understand its main organization. Plot is the specialization scaffold of the structure to one purpose; it is the basis for narrative and the most detailed part of planned structure. It is upon plots that pieces are rendered. Rendering is a particular way in which the plot was developed and filled with detail in order to be delivered to the audience. Remediation is the transformation of a creative piece already rendered into another one, re-rendered, possibly into another media. We now discuss some examples of this model in different creative disciplines. Music. If we consider a piece of music, for example, a composer can be innovative in the structure: a new form of concert, symphony or even something not known to this day; she may also be innovative in the plot: a new score, that is a new piece of written music; a new concert, for instance. Musicians can also be innovative in the rendering of the piece. That is the execution of the score with the realization of all the details needed to deliver the piece to the audience: a performance. Or they can also do remediation: transcribe an already composed piece of music from a string quartet to a rock band, for example. Literature. Here the structure refers to the genre. The most general structure of texts: tragedy, satire, comedy, etc. Plot is the structure of a particular story and rendering is the process to transform a plot into a complete literary piece that the audience can read. The rendered piece can also be subject to a process of remediation and be adapted to cinema or theater, etc. Performing arts usually concentrate on elaborating different renderings for given plots (scores). Each staging of a play in a theater is, in the terms used here, the rendering of a plot. That is, the specification of all the details needed for the audience to receive the original idea. If the performance is improvised, then the performers create, at the same time, both plot and rendering for the audience. Visual arts. In painting, we can consider the plot to be the sketch on a canvas, the initial drawings where the composition is outlined and the main elements designed. The rest of the work has to do with filling the details to complete the painting: details, colors, texture, etc. This is what we call the rendering. In this context, the structure aspect of the painting is its general description as a piece of art: oil on canvas. The audience perceives, in first instance, the rendering, then the plot and finally the structure. They go from the most emotional aspect of the set, to the most intellectual or logical one. Remediation may or may not be part of the piece, it is only included when a certain translation from media to media is needed. The rendered piece produces emotion in the audience while the plot produces understanding of the design behind the piece. Plot and structure enable communication, rendering and remediation, expression. All aspects are present in a piece in different degrees: in some works of abstract art like Jackson Pollock’s paintings, plot plays a minimal role, rendering is the most important aspect, there is hardly any structure, the emotion produced by the lines and colors is what constitutes its main expressive motif. In some pieces of conceptual art, on the other hand, structure and plot are the most important aspects while rendering is not as important. In Gabriel Orozco’s Cats and Watermelons, the number, order, size or disposition of the cans or the fruits (rendering) is not as important as the idea behind it all, the plot. One important thing about these four aspects of creativity is that they are not stages in the creative process, but they emerge during such a process, in any order or simultaneously. These properties might be the result of creative activity of and individual or a collective instance and they influence each other. Distinction between each, can characterize different forms of creativity. An art piece puts more emphasis on rendering while Design does it in plot. Literature and visual narratives strive to attain a balance between the two in order to maintain the equilibrium between clarity and expression to be enjoyed by an audience (McCloud, S. 2006). The Role of the Computer in a Creative Team Computers and computer programs are often used in creative processes. They can be used to store information, as tools, as means of displaying work, and many more. But not all of them have the same degree of creativity. We therefore distinguish five roles a computer program can play in a creative process: Environment. The computer is a medium where other members of the team can store, display, transmit and, in general, act as an environment where the work is created. Toolkit. The computer is used by members of the team, as a set of tools to transform and shape the work creation. Generator. The computer has been programmed to generate specimens or prototypes of partial or complete pieces of work that meet correctness rules. That is, the specimens belong to the desired kind (chairs, paintings, sonatas, stories, etc.) and team members can adjust parameters in order to vary the specimens generated. The final piece of work is either selected from the set generated or it is an elaboration of some elements of the set. Apprentice/assistant. The computer produces a reduced set of prototypes that, besides being correct members of the desired kind, they also fulfill some of the properties of creative products: e.g. valuable, innovative, surprising, etc. In this case, other team members have to choose the best of the candidates proposed by the system according to some more subjective human criteria (e.g. trendiness, politics, commission requirements, etc.). Master. The computer produces a complete and finished work that is considered creative by the designated experts. The rest of the team does management and configuration of the system and handling of the finished work. The environment role is the most common use of computers for creative purposes. Many people performing creative tasks have found that computers provide them with a suitable environment to work digitally on their subject matter. Working within a computational environment is often simpler, cheaper and more efficient. Another common role for computers is that of toolkit. Programs like Photoshop and many more like it that provide a set of tools a user can apply interactively and see the work progress are also ever more popular amongst artists. These systems have become indispensable for artists and creators and many activities like photography have already integrated tools like Photoshop into the basic set of tools for the profession. Many sophisticated systems apply a set of well-studied rules to produce correct pieces of work. These works are easily identifiable as part of the desired kind (poem, tale, motet, sculpture, etc.). It is useful to develop systems like these because they raise the level of abstraction in the process of creation. The programs generate works that can be considered candidates (or nearly) for a final. People using the system modify its parameters in order to alter the generation process and thus obtain better specimens. The user can be subtracted from the problem of assembling a product and concentrate on a new process by which the machine assembles the product and the user considers whether it is good enough or it needs to be modified somehow. Works produced by a generator system may be novel to itself, but not necessarily to the rest of the world. As we've said before, it takes a human eye to tell. Yet the generation process may expedite the overall process by speeding up a trial and error cycle. An apprentice system is one that has managed a new level of sophistication by showing a degree of knowledge that produces work specimens that fulfill general criteria for creativity (e.g. valuable, innovative, and surprising). Perhaps going from generator to apprentice is the challenge most computational creativity systems are facing in recent days. It can be seen as a search problem: moving from trial-and-error up to informed search methods. This last level is set as a reference. In the upper limit, a system that does all the important work and delivers a finished product that can be ascribed to a creative process is the ultimate capacity a computer system can acquire. We find several advantages to the model just described for the development of computational creativity: 1. A machine embedded in a creative process ensures that any development of the programs in it can be checked to see how much impact it has on the overall creative process. In particular, it is possible to verify whether the outcome of the whole process is still creative, thus eliminating the problem of generalizing toy worlds. 2. Versions of programs can be benchmarked according to the roles they are expected to play. 3. A staged plan for a research program can be drawn with clear goals and strategies based on roles assigned to participants. 4. The four aspects of creative products we described help teams to identify, for a particular role, what it is trying to achieve and decide how to evaluate its performance. Evaluating Creativity as Participation We have just described a framework that, we believe, can be used to assess creativity in a computational system by using it combined with already known methods from other fields like Design and applied Arts. These fields use participative and integrating methods to find out what is desirable and valuable for people (Ranjan, P.M. 2013). Participatory approaches are about including participants in the process of creation of product or experience. Evaluation aspects in this kind of projects are needed to measure impact and performance in the roles participants play. Nina Simon has developed a way to evaluate impact of participation of visitors in the context of museums and we think is relevant for our task. Her method consists of three main steps: 1. Stating the project goals. 2. Defining behaviors and outcomes that reflect those goals. 3. Measuring or assessing incidence and impact of the outcomes via observable indicators. Based on Simon’s model and using the apprentice framework described above, we can know how to proceed to either develop or assess a creative computational system. We should start by identifying which aspect of creativity is being emphasized, by doing so we are setting constrains and framing the context to work. This would drive the statement of project goals. Then we need to identify a particular role and skill that the computational system is expected to have by taking explicit knowledge from the humans members. Setting the skills and responsibility of the computer in the overall creative process would be the criteria for constant evaluation and modification of the system. It is important to stress that participatory projects often benefit from incremental and adaptive measurement techniques. Many creative outputs are process-based. So they have to be valued many times and incrementally before the project ends so that they stay aligned with the goals and all those involved are satisfied. (Simon, N. 2012).